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Simulation is introduced in terms of its different forms and uses, but the focus on discrete event modeling for systems analysis is dominant 
as it has been during the evolution of the technique within operations research and the management sciences. This evolutionary trace of 
over almost fifty years notes the importance of bidirectional influences with computer science, probability and statistics, and mathematics. 
No area within the scope of operations research and the management sciences has been affected more by advances in computing technology 
than simulation. This assertion is affirmed in the review of progress in those technical areas that collectively define the art and science of 
simulation. A holistic description of the field must include the roles of professional societies, conferences and symposia, and publications. 
The closing citation of a scientific value judgment from over 30 years in the past hopefully provides a stimulus for contemplating what lies 


ahead in the next 50 years. 


1. INTRODUCTION 
1.1. Background 


In this paper we present joint perspectives on the devel- 
opment and evolution of simulation over approximately 
50 years, recalling a few personal experiences during this 
period. We preface these remarks with a brief sketch of 
our introduction to simulation as graduate students during 
the early and mid-1960s. During the intervening period, 
both of us have been active in research and the teaching 
of simulation, heavily involved in service to the simula- 
tion community, and engaged with practitioners through 
consulting and sabbatical appointments. 

RGS wrote his first simulation program in 1961 using 
the Michigan Algorithmic Decoder (MAD) language while 
doing his graduate work at The University of Michigan. 
His principal appointment at Syracuse University was in 
industrial engineering and operations research, with addi- 
tional appointments in computer and information science 
and in electrical and computer engineering. His research, 
sponsored primarily by the U.S. Air Force over some 25 
years, focused on modeling and simulation. 

REN wrote his first simulation program in FORTRAN in 
1965. He used simulation to study the machine interference 
problem for his M.S. research at North Carolina State Uni- 
versity, which led to his interest in time flow mechanisms 
and the influence of model representation on execution 
efficiency. His Ph.D. dissertation at Purdue University is 
believed to be the first attempt to apply system dynamics 
to not-for-profit service organizations (university libraries). 
His permanent academic appointments have been in com- 
puter science/operations research and in computer science, 
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at Southern Methodist University from 1968 to 1973 and 
since then at Virginia Tech. He directs a research center 
established in 1983 by the U.S. Navy and has personally 
conducted projects in modeling and simulation, software 
engineering, and computer networking. 


1.2. Categorizing Simulation 


An understanding of the evolution of simulation is assisted 
by applying categorizations according to various criteria. 
One such categorization is based on the objectives of the 
simulation study. By far, the early work in simulation and 
that which has been dominant in management science and 
operations research over the history is system analysis, 
where the intent is to mimic behavior to understand or 
improve system performance. A second objective is educa- 
tion and training, where the former addresses the broader 
understanding of concepts and the latter, more specific 
behavior in the application of concepts. A third objective 
is acquisition and system acceptance, where the simulation 
model is intended to answer questions related to “Does the 
system meet the requirement?” or, “Does a subsystem con- 
tribute significantly to the improvement of the larger system 
performance?” A fourth objective relates to research which 
can involve the creation of an artificial environment. In 
such an environment, systems components can be tested or 
the behaviors of an individual or groups can be compared, 
contrasted, or categorized. Entertainment is the most recent 
objective: using a simulation model in a real-time interac- 
tive mode to derive pleasure and enjoyment. 

A second categorization relates to the representation of 
time and state in a simulation model. A Monte Carlo model 
requires state sequencing but no explicit representation 
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of time. Discrete event models specify state changes at 
discrete points in time. Continuous simulation portrays 
state changes as continuous over time, and discretized 
approximate solutions of differential equations are the 
most common examples. Combined discrete event and 
continuous models enable both techniques to be applied 
within the same study. Hybrid simulation models generally 
incorporate an analytical submodel within a discrete event 
model (Shanthikumar and Sargent 1983). 

Related to simulation models are games and gaming, a 
topic of considerable interest in the early history. Stim- 
ulated by the entertainment objective, games are experi- 
encing a strong resurgence, but the earlier batch mode of 
play is now replaced by real-time interaction with human 
players. (Games pitting computer programs against each 
other have also been staged.) 

The use of simulation precedes computers, either ana- 
log or digital. Described by some authors as “artificial 
sampling,” a manual Monte Carlo method was employed 
by Buffon to estimate m in a study documented in 1777 
(Jansson 1966). Hammersley and Handscomb (1964, p. 7) 
identify “Student” (a pseudonym) as using artificial sam- 
pling to calculate exact expressions for the distribution of 
the sample correlation coefficient and to derive what is now 
called Student’s t-statistic. 

Computer simulation began during World War II in the 
case of the continuous and Monte Carlo models. Discrete 
event simulation probably originated in the late 1940s; 
however, we have no evidence as to the exact date. During 
the remainder of this paper we focus on computer simu- 
lation with discrete event models and simply use the term 
“computer simulation” or “simulation.” 


2. THE EARLY DAYS 


Simulation books published in the 1960s present a rather 
uniform set of steps for conducting a study: problem 
formulation, system data collection and conceptual model 
formulation, validation of the conceptual model, construc- 
tion of the simulation program, execution of the simula- 
tion program, operational (results) validation, experimental 
design, output data analysis, and documentation. To give a 
feel of how the earliest steps were accomplished, we exam- 
ine the form of input, the content of the input, the execution 
of the program (running the model), and the output. 

The early simulation program consisted of a model 
description and an auxiliary set of simulation functions, 
including a random number generator, random variate gen- 
erators, list processing routines for queue insertion and 
deletion, a time flow mechanism, forms of model data col- 
lection and analysis, and a report generator. A Simulation 
Programming Language (SPL) representation of the model 
would include a library that the translator would access to 
provide these functions. If programmed in a General Pur- 
pose Language (GPL), the simulation modeler would have 
to rely on a GPL library routine or include a program to 
perform each necessary function. 


The content of the model, referred to as the model spec- 
ification, requires a world view (Weltansicht), as Lackner 
(1962) notes. (A referee notes that Weltanschauung is the 
correct term.) The early works of Lackner (1962) and 
Krasnow (1967) identify differences in world views that 
distinguish SPLs. Kiviat (1969) explores the similarities 
and differences in the first work to explain fully the SPL 
implementations of the world views. 

The implementation step (programming) of the model 
relied on simple techniques for creating uncertainty (ran- 
dom number and random variate production), about which 
little in terms of randomness properties was actually 
known. Arrays were used for simple list processing, and 
variable- or fixed-time incrementing was used in the time 
flow mechanism. Data collection methods used simple sta- 
tistical accumulations, again relying on the array data struc- 
ture. A deck of 80-column (also called “IBM”) cards rep- 
resented both program and data, and RGS recalls that at 
Michigan a typical turnaround time for computer jobs sub- 
mitted by graduate students was one to two days. Debug- 
ging of programs was often tedious and not infrequently 
required the decoding of “core dumps” with values in 
octal or hexadecimal representation. Storage limitations and 
costly processor time required conservation on both fronts. 

Output analysis often took the form of multiple repli- 
cations with different random number streams or simply 
reliance on an estimation of the mean without concern for 
variance estimation. Report generation was limited to a 
rather restricted set of output variables; however, early ver- 
sions of the SPLs did supply some rudimentary forms of 
dynamic error checking. 

The teaching of simulation in the 1960s was inhibited 
by the lack of textbooks. Often based on experience from 
practice, instructors used techniques that had no identify- 
ing source, and students were expected to know (or to 
learn) basic fundamentals of computer programming, list 
processing, and statistics as needed. RGS notes that sim- 
ulation was introduced as part of his course on data pro- 
cessing, required of both undergraduates and M.S. students 
in industrial engineering at The University of Michigan. A 
graduate course on simulation was also available, and typi- 
cal techniques included model specification by flow charts, 
an event scheduling world view representation, and imple- 
mentation in the MAD language. 

With instructional responsibility for simulation at Syra- 
cuse University in 1967, RGS used Naylor et al. (1966) as 
the text and taught both the event scheduling world view 
(using FORTRAN) and the transaction world view (using 
GPSS). In teaching his first course in 1968, REN exer- 
cised the same choices in languages and text. Programming 
simulations in the early period, irrespective of language, 
required the use of a manual provided by the language 
vendor. 

Excitement and expectation characterized both the aca- 
demic and industrial sectors of the simulation community in 
the 1960s. SPL developers were interested in sharing ideas 
and understanding different approaches, and this interest 


is reflected in a number of conferences and papers com- 
paring simulation languages. The first book on simulation 
appeared (Tocher 1963), and a number of others followed 
thereafter. REN and RGS believe that a healthy tension 
existed between research and practice, and the methods and 
techniques for modeling and simulation created during that 
period have had a lasting influence. 


3. TECHNICAL FACTORS IN 
SIMULATION DEVELOPMENT 


Examining the first few pages of a contemporary simula- 
tion book in comparison with one published 30 years ago, 
one notices strong similarities in the steps described for 
performing a simulation study. Essentially, what charac- 
terizes the modeling and simulation activities today seems 
little different; however, how those steps are performed 
differs considerably. We have chosen to separate factors 
marking the evolution of simulation into two categories: 
(1) external—those emanating from computing technology 
that set directions or shaped the progress of simulation 
research and practice; and (2) internal—those generated by 
the communities of simulation researchers and practition- 
ers. Within each factor below, the development is described 
in a loose chronological order. We readily admit that selec- 
tion of these factors is subject to personal biases; others 
might identify influences felt to be more significant and 
could present strong arguments for their choices. 


3.1. External Factors Shaping the Evolution 


The early pervading view of simulation as a problem- 
solving technique stems from the development of Monte 
Carlo techniques well before the appearance of either ana- 
log or digital computers. Monte Carlo computations per- 
formed on electromechanical calculators by a host of oper- 
ators was the common solution procedure in the 1950s 
for numerical models (approximate solutions of differential 
equations). Yet simulation, and in particular discrete event 
simulation, could never have been a major problem-solving 
technique without the emergence and rapid development 
of the digital computer. Consequently, our view is that the 
external influences on simulation are dominated by those 
associated with digital computing technology. 


3.1.1. The Revolution in Computer Hardware. Youth- 
ful faces and sprightly gaits aside, we are made even more 
conscious of the huge gaps in computing history sepa- 
rating us from our students when we realize that most 
today do not recognize the terms: “mainframe computer,” 
“core memory,’ or “keypunch.” Since the ENIAC in the 
late 1940s, progress in computer hardware has advanced 
at a revolutionary pace. Processor speeds and storage sizes 
(both primary and peripheral) have increased by several 
orders of magnitude, while component size has decreased 
to a like degree as the succession of hardware technologies 
has transitioned from the mainframes of the 1960s to the 
minicomputers of the 1970s, the parallel processors of the 
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1980s, networks of processors in the 1990s, and the desk- 
tops and laptops joined by wireless connections of today. 
While the time intervals stipulated above are imprecise, the 
impact on modeling and simulation has been pervasive. 

For the vast majority of the OR community, recognition 
of the hardware influence needs little justification, but we 
believe that the effects are more pronounced for simula- 
tion than for most areas. That claim aside, the incredible 
advances in computer hardware must be acknowledged as 
making simulation a viable problem-solving technique for 
some and the preferred technique for many. 


3.1.2. Advances in Computer Software. The machine 
language representations of the early 1950s gave way to the 
assembly language of the mid-1950s. The improved rep- 
resentational capability, supporting the list processing and 
functional library organization needs for simulation, made 
a significant contribution to development of the first pack- 
aged simulator: the General Simulation Program (GSP) of 
Tocher and Owen (1960). By the late 1950s FORTRAN had 
extended the semantics helpful for understanding model 
representations. While FORTRAN emerged as the domi- 
nant language for engineers and scientists in the United 
States throughout the next two decades, its limited data 
structures (the array) in the early versions had some lasting 
effects. FORTRAN simulation packages such as GASP and 
MILITRAN provided functional capabilities, but in general 
inhibited the acceptance and widespread use of SPLs that 
were appearing in the mid-1960s. 

In Europe ALGOL was the dominant language, and its 
failure to achieve widespread acceptance in the United 
States was in part due to a hardware influence: the domi- 
nation of IBM in the mainframe market at the time. SIM- 
ULA 67, as an extension of ALGOL, ushered in the object- 
oriented programming style. Popularized by Smalltalk in 
the 1980s, object-oriented programming would become the 
dominant software methodology in the 1990s. 

A burgeoning interest in SPLs stimulated a number of 
representational issues related to specification and abstrac- 
tion in model development. Graphical representations were 
prominent in some early languages, notably the flow 
chart symbology in GPSS and the Activity-Cycle Diagram 
(ACD) (or wheel chart) in the Control and Simulation Lan- 
guage (CSL), popular in the United Kingdom. The his- 
tory of simulation programming language development is 
described briefly in Crain et al. (1992) and by Hixson in 
Araten et al. (1992). A more detailed description of the his- 
tory of SPLs from 1955 to 1985 is given in Nance (1996). 

The recognition of software engineering as an area 
of study had its own effects on simulation in the late 
1960s and early 1970s. Model documentation, stimulated 
by issues in program documentation, became an impor- 
tant concern as did the life-cycle perspective and the 
user involvement in model development. Two government 
reports (U.S. General Accounting Office 1973, 1976), iden- 
tified major deficiencies in “computerized models” (many 
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were simulation studies). Sensitivity to good software engi- 
neering practices became a requirement for major simula- 
tion modeling efforts. Arguments were advanced that model 
representation should generate model documentation, and 
that the common consideration of documentation as an 
after-the-fact “activity” was a major detriment to effective 
use of the model (Nance 1979). 


3.1.3. Influences by Other Computing Technologies. 
The influences cited in this section derive from technical 
areas that do not fit within either hardware or software. A 
brief description serves to support the assertion that these 
technical areas have had notable influence on modeling and 
simulation. 

3.1.3.1. Computer Graphics. Utilizing the capabilities 
to discriminate based on color, perspective, and motion, 
advances in computer graphics have led to the use of 
animation for model output and increased the credibility 
of simulation results. Interestingly, the early emphasis on 
graphical input (the flowchart symbols in GPSS and the 
Activity-Cycle Diagrams) did not persist with the major 
advances in the 1980s. An early development that never 
reached commercial use was the RAND Tablet that trans- 
formed the drawing of GPSS flow-chart symbols automat- 
ically into a GPSS program. GPSS/NORDEN (Reitman 
et al. 1970) demonstrated the use of output animation using 
vector graphics. The NORDEN version also provided a 
graphical depiction of transaction queueing in the GPSS 
flow diagram to assist in program debugging, which, cou- 
pled with the capability for user interrupts, permitted some 
interactive corrections and changes. 

Graphical interests in the late 1970s and early 1980s cen- 
tered on output animation. Color and motion were promi- 
nent in depicting product transformations during execu- 
tion of the simulation model. Visual Interactive Simulation 
(VIS) became a prominent technology in the mid-1980s, 
and the claim was made by O’ Keefe (1986) that the United 
Kingdom was ahead of the United States at that time in 
this technology. Bell and O’Keefe (1994) describe VIS in 
methodological terms, contrasting the active and passive 
forms in model development and experimentation. Cur- 
rent work in graphics is pushing the development of three- 
dimensional displays of output behavior. 

3.1.3.2. Human-Computer Interaction. Significant 
developments in human-computer interaction (HCI) would 
not have been possible without hardware and software 
advances that enabled time-sharing operating systems and 
interactive programming. HCI has the goal of making 
interactive software efficient, effective, safe, and satisfying 
in its use (Hartson 1998). A major consequence of the con- 
junction of HCI with other advances is an ever-increasing 
user relief from the requirement to have detailed knowl- 
edge of the underlying computing technology. The result 
has greatly expanded the population of productive users of 
the ubiquitous digital technology. However, a concomitant 
result is that, unless the user forces revealing actions, the 
modeling software hides how the function is performed. 


An unsettling consequence is that simulation model users 
need not be those who developed the model, and users are 
likely to have little understanding of how the model results 
are being produced. Furthermore, model developers some- 
times lack a sufficient understanding of the internal logic 
of SPLs to enable the recognition of erroneous results 
produced by incorrect models. 

3.1.3.3. Computer Networks. The dumb terminal inter- 
face enabled by time-sharing systems of the 1970s, and 
first used for simulation purposes by the OPS project at 
MIT in the 1960s, was displaced by the networking of ter- 
minal interfaces in the 1980s. Several terminals connected 
to a minicomputer, supplemented by networked commu- 
nications among several other minicomputers, was a typ- 
ical architecture. As the minicomputers gave way to the 
microprocessor workstations, the growth of networks accel- 
erated. The National Bureau of Standards (now National 
Institute of Standards and Technology) was a major factor 
through its leadership in the international arena that led to 
the creation of local area and metropolitan area network- 
ing standards. The preponderance of networked computing, 
coupled with the emergence of the Internet, provided the 
enabling factors for distributed interactive simulation. 

3.1.3.4. The World Wide Web. In this past decade, net- 
work computing has expanded to a global level. Web-based 
simulation is now an implementation issue rather than a 
research concept. The potential in web-based simulation is 
for a model to be constructed and provided as a commod- 
ity. Users can define a set of parameter values and select 
alternate structures internal to the model in configuring an 
experiment. Remote execution is invoked to produce the 
simulation output. The maintenance and modifications are 
left with the model producer, and the simulation activity 
assumes the role of receiving service from a utility. The 
responsibilities of model producer and simulation experi- 
menter are clearly distinguished. 


3.2. Internal Factors 


Selecting the key factors in over 40 years of research and 
experience in simulation is a challenge. We offer our sug- 
gestions in the brief description that follows. 


3.2.1. Modeling. As the SPLs of the early 1960s 
emerged, each offered a conceptual framework derived 
from an application area, the influence of a GPL, or some 
combination of influences. Amidst the din of claims from 
the language disciples, a few such as H. S. Krasnow (1967) 
sought to fathom the SPL differences in more fundamental 
language-independent terms. He categorized world views 
for continuous, discrete, and combined (discrete and con- 
tinuous) simulation and described ways of representing sys- 
tems for discrete simulation in a paper presented at the 
1965 NATO Conference on Digital Simulation in Opera- 
tional Research. The representations for discrete event sim- 
ulation included the event, activity, and process represen- 
tations. A detailed comparative analysis of the differences 


in these three, and the transactional, world views are con- 
tained in Kiviat (1969). 

Graphical assistance in model specification accompa- 
nied the introduction of the languages for the transaction 
(GPSS) and activity (CSL) world views. For the latter, the 
activity-cycle diagram served several SPLs and was the 
basis for the interactive program generation work in the 
United Kingdom in the 1970s. Event Graphs were intro- 
duced by Schruben (1983) to assist in model building using 
the event world view and further developed by Sargent 
(1988) and Som and Sargent (1989). (We note that a repre- 
sentation similar to event graphs was introduced in Evans 
et al. 1967, but apparently not further developed.) 

A graphical model representation for the process world 
view, called Control Flow Graphs, was developed by Cota 
and Sargent (1990) (see also Cota et al. 1994) and later 
extended to Hierarchical Control Flow Graphs by Fritz and 
Sargent (1995) to aid in the control of representational com- 
plexity. 

Model development environments that were research 
subjects in the 1980s, see Nance (1983), have become the 
practitioner’s initial modeling tool in the 1990s. In 1990, 
the renaming of the tutorial track “Software” to “Software 
and Modelware” in the program of the Winter Simula- 
tion Conference reflected the expansion of modeling tools 
beyond the SPL level. Today, visual interactive modeling 
employs icons, graphical depictions, or actual pictures of 
system elements imported to provide a more recognizable 
association with the system counterpart. Modeling method- 
ology, which includes events list management, automated 
and semi-automated modeling techniques (diagnosis, agent- 
based approaches), time flow mechanisms, and validation 
and verification, is recognized as a primary research area. 
The transition to environments has relieved most practition- 
ers from direct involvement with modeling methodology 
issues, relegating them to provided functions. This indirect 
involvement does exact a price: the inability of model users 
to recognize potential errors (see §3.1.3.2 above). More- 
over, these simulation functions remain as essential con- 
tributors to the success of a simulation study. 


3.2.2. Simulation Functions. 

3.2.2.1. Random Number and Random  Variate 
Generators. Random number generation (RNG), tests for 
randomness, and transformation techniques (random vari- 
ate generators) have been active research topics since 
the advent of the digital computer. The congruential (or 
Lehmer) generators (Lehmer 1951), whose behavior could 
be based on number-theoretic properties, displaced the 
ad hoc techniques in the 1960s, but various unsubstan- 
tiated methods for achieving better randomness proper- 
ties can be found in publications into the 1980s. Knuth 
(1997) provides a comprehensive description of RNG tech- 
niques and tests for randomness, and descriptions of current 
research findings are provided by L’Ecuyer (1998, 2001). 
The field of random variate generation gained maturity in 
the 1980s after much research in the prior decades. An 
extensive treatment of transformation techniques is given 
in Devroye (1986). 
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3.2.2.2. Time Flow Mechanisms and Event List Manage- 
ment. Early simulation programs used either a fixed-time 
increment (FTI) or variable-time increment (VTI) method 
as the basis for control of time. The developer of an SPL, 
influenced by application area or perception of run-time 
efficiency often chose between FTI and VTI inevitably cre- 
ating the world view implemented in the language. Each 
world view promotes a particular way of characterizing 
the relationships among model objects and their attributes 
depicting time and state (Nance 1981). 

The event view imposes the implementation of time 
passing because events occur, and event list management 
(insertion, reordering, and removal) determines the execu- 
tion time for a simulation model. A contentious issue in the 
1960s was the comparative performance of FTI and VTI 
methods. Nance (1971) showed that universal superiority 
could not be claimed by either method. 

During the late 1970s and early 1980s research in data 
structures for event list management received major atten- 
tion. Personally involved in this work, RGS recalls the 
excitement on discovery that efficient list processing algo- 
rithms could reduce the run times of some simulations up 
to 30 or even 40% (McCormack and Sargent 1981). 

The three-phase extension to the activity scan method, 
attributed to Tocher, dominated in the United Kingdom and 
the SIMULA co-routine implementation of process interac- 
tion had major influence in Europe and a few United States 
locations. Both of these methods can transition between 
resembling a next event or an activity scan method. The 
process view treats the object as primary, but the (process) 
transactional view characterizes only the dynamic objects 
as processes. (Hence, the term “active resource” is also 
used to distinguish the “pure” process view from the trans- 
action view.) 


3.2.3. Verification and Validation. The importance of 
basing a decision on results from a valid model is under- 
scored in an early text, Naylor et al. (1966), that quotes an 
earlier paper containing a definition of simulation proposed 
by C. West Churchman (1963): 


“X simulates Y” is true if, and only if, (a) X and Y are 
formal systems, (b) Y is taken to be the real system, (c) 
X is taken to be an approximation to the real system 
and (d) the rules of validity in X are non-error-free. 


Thomas Naylor, a coauthor of the book cited above, 
deserves credit for drawing major attention to the valida- 
tion issue in the 1960s: Is the model actually represent- 
ing the truthful behavior of the referent system? His work, 
above and in later publications (Naylor 1971, Naylor and 
Finger 1967), exerted a major influence in framing valida- 
tion within different philosophical perspectives. Numerous 
techniques that can be used were identified or developed. 
While the issues of both verification and validation were of 
concern from the early days of simulation, often no clear 
distinction was made between the two terms. 

In the late 1970s Sargent (1979, 1981) and Balci and 
Sargent (1980, 1981) raised the visibility and understand- 
ing of verification in contrast with validation and placed the 
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latter within a sound statistical framework. With the strong 
interest in verification from the software engineering com- 
munity, this contrasting but complementary explanation of 
the term was quite important. The effort to place valida- 
tion in a cost-risk framework moved the concept from a 
philosophical explanation in earlier works to a form more 
useable for simulation practitioners. 

Current views hold verification and validation to be 
separate processes, each employing techniques appropriate 
to the differing objectives. Informally, verification focuses 
on the activities in developing the model (“producing the 
model correctly”) and validation focuses on comparison of 
the model with the referent system (“producing the cor- 
rect model”). Formal statistical tests developed for model 
validation are difficult to apply in practice because of the 
required assumptions and/or availability of system data. 
In practice, both verification and validation are often per- 
formed using subjective (inspection) approaches, with the 
validation being given the greater attention. 


3.2.4. Analysis Methodology. Richard Conway (1963) 
initiated a research area that became characterized as anal- 
ysis methodology. This paper was the first to take a holis- 
tic approach to simulation experimentation, identifying the 
two phases as “strategic planning” and “tactical planning.” 
While this paper concentrates on tactical planning, in par- 
ticular the difficulties inherent in steady-state parameter 
estimation, it also discusses the use of variance-reduction 
techniques and different statistical approaches for compar- 
isons of alternatives systems (or operating policies) using 
simulation. The analysis methodology area has been and 
continues to be an extremely active research area with 
papers on the subject numbering in the hundreds. (See Law 
and Kelton 2000 for a detailed discussion of current anal- 
ysis techniques.) 

3.2.4.1. Output Analysis. The analysis of simulation 
output is divided into two system classes: steady-state and 
terminating. Systems such as banks and many retail outlets 
can be modeled as terminating simulations if replications 
of a defined operating period can be assumed to consti- 
tute an independent and identically distributed random sam- 
ple. The classical statistical analysis techniques can then be 
employed. 

Systems modeled for steady-state analysis introduce the 
complexities of: (1) removal of the bias of the imposed ini- 
tial model state and (2) definition of a sample that admits 
an accepted estimate of sample variance, which is needed to 
determine the precision of estimates of steady-state param- 
eters. Removal of initial state bias (also called the ini- 
tial transient problem), despite some innovative approaches, 
remains an unresolved problem (unless the regenerative 
process technique is employed). Estimation of sample vari- 
ance by imposing assumptions to apply the method of 
replications or the batch means method (dividing one long 
series of output values into batches with the autocorrelation 
among them included in the variance estimate) are both 
addressed by Conway (1963). Theoretical and experimen- 
tal research since that time has significantly improved the 


understanding of the behavioral properties of both meth- 
ods. Developing procedures to determine appropriate batch 
sizes remains an active research topic (see papers in recent 
issues of the Proceedings of the Winter Simulation Confer- 
ence). 

Research in other approaches for variance estimation 
include spectral analysis, autoregressive models, regenera- 
tive processes, overlapping batch means, standardized time 
series, and combinations of different methods. Alexopoulus 
and Seila (2000) provide an instructive description. Com- 
parative behaviors have been investigated using theoretical 
and experimental approaches (for example, Sargent et al. 
1992 employs both), and each method has proponents. 
Generalized Semi-Markov Processes (GSMPs) have been 
proposed as a foundation for steady-state output analysis 
(Glynn 1989, Haas 1999). Variance estimation for steady- 
state analysis remains an actively investigated problem. 

While the discussion above dwells on the estimation of 
the mean and associated confidence interval for a single 
simulation model parameter, active research continues on 
other output analysis techniques. Included in this group 
are quantile estimation, multiple joint measures, the use 
of fixed-sample-size versus sequential-sample-size proce- 
dures, Bayesian statistics (Chick 2000), jackknife and boot- 
strap sampling (Efron 1982, Efron and Tibshirani 1993). 
Variance-reduction techniques, described below, have also 
received much attention. 

3.2.4.2. Experimental Design and Comparison of Alter- 
natives. From the early 1960s until today, considerable 
research has dealt with the use of classical design of exper- 
iments in simulation for such applications as compari- 
son of alternatives, metamodeling, optimization, sensitiv- 
ity analysis, and validation (see Kleijnen 1975). Conway, 
in his 1963 paper, suggests that the newly proposed (at 
that time) ranking and selection (R&S) procedures were 
more appropriate for comparison of alternatives than tech- 
niques derived from the classical design of experiments. In 
the interim, considerable research has been conducted in 
R&S procedures in general, see Bechhofer et al. (1995); 
and specifically for simulation, see Goldsman and Nelson 
(1998). Software enabling the use of R&S procedures in 
simulation studies is now included in several commercial 
simulation products. The use of variance-reduction tech- 
niques with the design of experiments and with R&S pro- 
cedures is the subject of numerous articles. 

3.2.4.3. Metamodels and Optimization. Metamodels— 
(simple) mathematical models of the output response sur- 
face of a simulation model—have been studied in terms of 
both the metamodeling role (Kleijnen and Sargent 2000), 
and the types of models that can be used (Barton 1998). 

Optimization of simulation model output has a number of 
complexities: The number of model variables is often large, 
and a variable (or parameter) can take on a large or infi- 
nite number of values. Sometimes the response surface is 
multimodal. Various approaches have been suggested over 
the years, including the use of gradient-based optimization 


methods, response surface methodologies (including meta- 
models), pattern search methods, and random search. (With 
a finite number of alternatives specified, the R&S proce- 
dures discussed above are candidates.) Much of the atten- 
tion to these methods involves the convergence to a local 
or global optimum. The sophistication and computational 
intensity of these methods limit their use in practice. More 
information on optimization can be found in recent issues 
of the Proceedings of the Winter Simulation Conference. 

More recent research (1990s) has taken a different tack 
towards optimization: expressing the objective so as to 
obtain a “good” but not necessarily optimal solution. These 
approaches use some type of metaheuristics such as tabu 
search or a genetic algorithm. Several commercial simula- 
tion software systems today contain “optimization” pack- 
ages based on one of these approaches. 

3.2.4.4. Variance-Reduction Techniques. Variance- 

reduction (or reducing) techniques (VRTs) received much 
attention in the early days of simulation since computer 
time was extremely expensive, and reductions in run time 
represented valuable savings. The extensive use of VRTs 
in Monte Carlo studies suggested similar efficiencies in 
(discrete event) simulation. Using VRTs in the collection 
and analysis of data has been a focus. While a few VRTs 
are simple to use, most VRTs are sophisticated and model 
dependent, which limits their general use. A counterexam- 
ple is common random numbers, often found in commercial 
simulation products because they are easy to understand 
and simple to apply. The two cases where (sophisticated) 
VRTs are justified: (1) simulation models investigating rare 
events and (2) simulation models that exact excessive com- 
putation time. 


3.2.5. Theory of Simulation. As early as 1964 Lackner 
had proposed the use of system theory as a basis for sim- 
ulation modeling. First with a journal paper in 1972, then 
with his book in 1976, Zeigler built an explanatory the- 
ory of simulation based on systems-theoretic concepts. This 
work had a major impact on those who sought to separate 
the expression of simulation concepts from their implemen- 
tation in SPLs. The theoretical structure applied to discrete 
event, continuous, and combined models provided a link- 
age that heretofore was difficult for many to conceive. 


3.2.6. Factors Contributing Jointly to Simulation and 
Computer Science. Although originating in simulation, 
at least three concepts have had a major influence in areas 
of computer science and the advancement of computing 
technology. 

3.2.6.1. The Process Concept. Embodied restrictively 
in the GPSS transaction representation and expanded more 
elegantly in the SIMULA process interaction world view, 
the process concept is a lasting contribution to both sim- 
ulation and operating systems. Its influence in simulation 
was to provide the realization of an entity whose dynamic 
behavior it sought to mimic. In the operating systems con- 
text the process presented a quasi-independent program 
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segment in execution, and served as a major concept under- 
lying computational models. The process concept within 
the co-routine execution environment provided by SIM- 
ULA provided a powerful mechanism for expanding the 
ability to represent complex systems. 

3.2.6.2. The Entity/Attribute/Set Concept. Introduced by 
Kiviat et al. (1968) in SIMSCRIPT II, this modeling per- 
spective provided a rigorous basis for describing the static 
relationships among objects. Entities could be members 
of and owners of sets, yet each was described individu- 
ally by its own attributes. Coupled with the recognition 
of relationships among entities, captured so expressively 
by Mealy (1967), the underlying concepts of the entity- 
relational model of data were actually present in simulation 
for almost 10 years before their recognition by the database 
community. 

3.2.6.3. Object-Oriented Programming. The revision to 
SIMULA I, known as SIMULA 67, introduced the object- 
oriented paradigm (OOP) with concepts of abstract data 
types, encapsulation, inheritance, and message passing. 
The co-routine concept from the earlier version, enhanced 
by the OOP capabilities, promoted a very powerful style 
of simulation programming, so powerful that after two 
decades the OOP became the predominant style for pro- 
gramming in general. This particular factor has exerted an 
effect far beyond simulation, substantiated by the fact that 
four of the eight most significant languages selected for 
the 1993 History of Programming Languages II Confer- 
ence (HOPL II) (see Bergin and Gibson 1996) traced their 
major roots to SIMULA (Ada, C++, CLU, and Smalltalk). 
Despite its impact on programming languages in gen- 
eral, one of the SIMULA co-developers (Nygaard 1978) 
remarked to REN that only those who had used SIMULA 
as a simulation language truly understood its power. 


3.2.7. Combined Simulation. The evolution of GASP 
enabled combined continuous and discrete modeling in 
GASP IV through the work of Pritsker (1974). Pritsker and 
his students worked out the detailed transitional relation- 
ships between the continuous and discrete computations by 
adding the necessary subroutines to the earlier version of 
the language (GASP II) and providing an alternative defi- 
nition of the term “event.” (An interesting trivia note is that 
some early copies of the book had the title “The GASP IV 
Language Simulation” printed on the spine and the correct 
title on the front of the book jacket.) (Pritsker’s numerous 
contributions to simulation extended over a major part of 
its history (Wilson and Goldsman 2001).) 


3.2.8. Parallel Simulation. Initiated by research in the 
late 1970s and early 1980s by Bryant (1977), Peacock et al. 
(1979), Chandy and Misra (1979), and Jefferson and Sow- 
izral (1982), parallel simulation became a major area of 
research in the middle to late 1980s and extending into the 
1990s. Fueled by the time warp concept formulated by Jef- 
ferson and Sowizral (1982), research using this optimistic 
protocol in which checkpointing with rollback and recov- 
ery is required, was contrasted with the conservative proto- 
col of Bryant (1977) and Chandy and Misra (1979), where 


168 / NANCE AND SARGENT 


no events were executed unless correct temporal order- 
ing was guaranteed. A series of conferences bearing the 
name PADS (Parallel and Distributed Simulation) began in 
1985, and the Proceedings contains much of this research. 
Strong tensions between the proponents of the two proto- 
cols marked the early conferences, but now have all but 
disappeared. The intense early interest in the subject has 
also waned considerably. 

A special issue of the ORSA Journal on Computing in 
1993, guest edited by Richard Fujimoto and devoted to 
parallel discrete event simulation, raised the issue of why 
parallel simulation has not been accepted in the broader 
domain of simulation practice. Numerous answers were 
offered, both within and outside the PADS community, but 
no consensus has formed. 


3.2.9. Distributed Interactive Simulation. Enabled by 
network computing advances, the concept of distributed 
interactive simulation originated in the military domain for 
training, and the visionary concept is generally attributed 
to Thorpe, circa 1978 (see Cosby 1995). Major funding by 
ARPA/DARPA permitted the demonstration that remotely 
executing simulation models could communicate, although 
major questions still remain regarding the correct repre- 
sentation of temporal causality and trade-offs between the 
level of model fidelity versus the cost of the training exper- 
iment. The early Distributed Interactive Simulation (DIS) 
and Aggregate Level Simulation Protocol (ALSP) are being 
supplanted by the High Level Architecture (HLA) protocol, 
intended to enable interoperability among DOD simulation 
models. 


4. ORGANIZATIONAL FACTORS 
4.1. Conferences and Symposia 


Interest in the new field of computer simulation and excite- 
ment about future prospects was high in the late 1950s. 
Meetings provided the prime venue for communications. 
One of the first was the System Simulation Symposium 
held in 1957, to be followed closely by the Second Sym- 
posium on System Simulation in 1959. Another of note 
was the IBM Scientific Computing Symposium on Simula- 
tion Models and Gaming held in December 1964 with 175 
attendees. Workshops on simulation languages were held 
at Stanford University in 1964 and at the University of 
Pennsylvania in 1966, the latter having 110 attendees. The 
NATO-sponsored conference on digital simulation held in 
Hamburg in 1965 had 180 attendees. A symposium on “The 
Design of Computer Simulation Experiments,” sponsored 
by the The Institute of Management Sciences (TIMS) Col- 
lege on Simulation and Gaming, was held at Duke Univer- 
sity in 1968 (RGS attended along with 250 others). IBM’s 
SHARE User Group set up a System Simulation Project 
that held meetings where changes to GPSS were discussed. 
Three members of this group, H. Hixson, A. Ockene, and J. 
Reitman, feeling the need for a national conference on the 
applications of simulation, organized the “Conference on 
the Applications of Simulation Using GPSS” in November 


1967 in New York City. A planned attendance of 225 and 
an actual of 401 encouraged a successor, held in Decem- 
ber 1968. The follow-up conference was called “The Sec- 
ond Conference on the Applications of Simulation” (note 
the removal of GPSS); a proceedings was issued; and the 
attendance numbered 856. 

The two application-oriented conferences above began 
what is now called the Winter Simulation Conference 
(WSC). See Crain et al. (1992) and Araten et al. (1992) for 
a history of the WSC through its 25th anniversary in 1992 
and (http://www.wintersim.org/article.htm) for a regularly 
updated overview of the conference and its more recent 
history. The WSC, held each December, is the premier 
conference in simulation, attracting international attendees 
drawn from researchers, practitioners, and simulation soft- 
ware vendors. Considered a “model” conference, the WSC 
is sponsored by several societies, including INFORMS, and 
is run by volunteers. The conference attracts high qual- 
ity papers, publishes electronic and hard-copy proceedings, 
and offers exhibits by vendors. 

The Annual Simulation Symposium, initiated in 1968 by 
Ira Kay, is a single-track conference. Both REN, who pre- 
sented a paper at the 1969 meeting, and RGS, who attended 
in 1979, recall the very smooth and efficient operation of 
the conference. Operating now under Society for Modeling 
and Simulation International (SCS) sponsorship, the sym- 
posium lays claim to being the longest continuously run- 
ning simulation conference. 

In response to growing interest in the modeling of com- 
puter systems for performance evaluation in the 1970s, 
the Association for Computing Machinery (ACM) Special 
Interest Group on SIMulation (SIGSIM) and the National 
Bureau of Standards cosponsored a series of symposia 
(with proceedings) and workshops on the topic. RGS still 
recalls the enthusiastic response by attendees to his tuto- 
rial paper on the statistical analysis of simulation output 
data at the 1976 symposium. Specialized languages for 
computer systems simulation were developed during this 
period, and a Federal agency, FEDSIM with Philip Kiviat 
as director, was established for computer system perfor- 
mance improvement. 

Two conferences on simulation research were held by 
SICSIM of NYC (Special Interest Committee for SIMu- 
lation of New York City) under Nabil Adam’s leadership. 
Papers from the first led to the publication of a book, 
Adam and Dogramaci (1979), and those from the second 
to a special issue of Communications of the ACM (April 
1981). Adam, REN, and RGS organized a follow-up con- 
ference sponsored by ORSA and SICSIM of NYC that 
led to a special issue of Operations Research (November— 
December 1983). 

Tuncer Oren was extremely active in the organiza- 
tion of meetings in Europe, notably a 1979 workshop on 
the standardization of simulation languages in St. Agata, 
Italy. REN remembers observing Harry Markowitz read- 
ing Zeigler’s Theory of Modelling and Simulation during 
a conference break. Discussing the book later, Markowitz 


dryly remarked as to how unthinking of people to change 
the field on him during the past 15 years. 

François Cellier (1982) renewed the European model of 
the conference with the papers appearing in a book (as had 
the NATO conference proceedings earlier). Oren, Maurice 
Elzas, and Bernard Zeigler organized a series of four con- 
ferences following this model. RGS and REN, participants 
in some of these conferences, recall that several outstand- 
ing papers were presented, leading to interesting and lively 
discussions. 

Described above as areas of intense research activity, 
parallel and distributed simulation spawned a conference 
bearing the acronym “PADS” that since the mid-1980s has 
occurred (with proceedings) almost annually. Technically 
related to PADS is the Simulation Interoperability Work- 
shop (SIW), a semiannual meeting encompassing a broad 
range of modeling and simulation issues, applications, and 
communities (see http://siso.sc.ist.ucf.edu/siw/). 

A dozen or more conferences with simulation in the title 
or featuring simulation as a major topic are offered annu- 
ally, sponsored by numerous organizations. The Summer 
Computer Simulation Conference (SCSC) is a complemen- 
tary conference to the WSC. Sponsored by SCS and origi- 
nally limited to continuous simulation, the scope in recent 
years has expanded to include discrete event simulation. 
Several multiconferences are sponsored or cosponsored by 
SCS, with a typical format of concurrent one- and two-day 
miniconferences involving different technical and applica- 
tions topics. 


4.2. Professional Organizations 


Simulation draws its professional lifeblood from special 
interest groups within larger societies. The principal group 
today is the INFORMS College on Simulation (CS), 
founded in 1963 as the College of Simulation and Gam- 
ing (CSG) within TIMS. Another group is SIGSIM of 
ACM, formed in 1967 and extremely active in the 1970s 
and 1980s. Preceding both TIMS/CSG and ACM/SIGSIM 
was SCS, originally founded as Simulation Councils Inc. in 
1952 under the leadership of John McLeod. Originally lim- 
ited to continuous simulation, SCS now includes all types 
and forms of simulation. Within the Institute of Electrical 
and Electronic Engineers (IEEE) are the Computer Soci- 
ety and the Systems, Man, and Cybernetics Society. Both 
have strong interests in simulation, and the former has a 
subgroup with the title Technical Committee on Simula- 
tion (TCSIM). Simulation is also an area of technical and 
publication interest for the Institute of Industrial Engineers 
(IIE). 


4.3. Simulation Coverage in Journals 


Various professional journals established departments to 
handle simulation papers in the mid-to-late 1970s: AHE 
Transactions in 1976 with REN as editor, Management Sci- 
ence in 1978 with George Fishman as editor, Operations 
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Research in 1978 with REN as editor, and the Commu- 
nications of ACM (CACM) in 1980 with RGS as editor. 
(CACM transitioned from a focus on research papers to 
informative articles in the late 1980s.) When the ORSA 
(now INFORMS) Journal on Computing was established in 
1989, it contained an area on simulation with REN as its 
editor. 

SCS established an archival journal in 1984 called Trans- 
actions of the Society for Computer Simulation to handle 
both continuous and discrete event simulation. The jour- 
nal Transactions on Modeling and Computer Simulation 
(TOMACS), devoted primarily to discrete event simulation, 
was established by ACM in 1990 with REN as editor- 
in-chief. The current publication venue offers numerous 
archival journals where simulation papers can be published. 

From 1988 to 1994 Paul Fishwick maintained Simula- 
tion Digest, the first online publication devoted explicitly 
to simulation interests. Another online publication bearing 
the same name was launched during the 1988-1990 time 
frame jointly by Fishwick (Chair of TCSIM) and Stephen 
Roberts (Chair of SIGSIM) to serve as a joint organiza- 
tional newsletter. 


5. CONCLUDING SUMMARY 


In drawing this trace of the evolution of simulation to a 
close, comments on three aspects are offered. The first con- 
cerns the breadth and extent of simulation applications; the 
second relates to the differences in views of scholarly depth 
of simulation research; and the third pertains to the future 
of simulation. 


5.1. Applications: What Cannot Be Simulated? 


From the beginning, the ingenuity and innovation with 
which simulation is applied has proved impressive. The 
adoption of simulation in numerous fields has created a 
disturbing side effect as well: The term is often inappropri- 
ately used. An early book, Shapiro and Rogers (1967), pro- 
vides a fascinating collection of papers on self-reproducing 
systems, the use of graphics in studying dynamic system 
instability, associative processor design, simulation of the 
human aorta for studying artificial blood pumps, design of 
a parallel network computer, torpedo performance analy- 
sis, and so forth. A common misconception was, and still 
persists, that any computational process produces a “simu- 
lation.” Nevertheless, proper applications of the technique 
have abounded. 

A collection of papers on simulation in the social sci- 
ences, economics, business, and public administration, with 
a foreword by Herbert Simon, is given in Guetzkow et al. 
(1972). Models ranging from neurotic processes of an indi- 
vidual to the interaction of nations in producing the events 
leading to World War I are described. Bonini’s (1963) 
aggregative model of a hypothetical firm is a stark con- 
trast with the systems (nee industrial) dynamics models of 
Forrester (1961). Cognitive process simulation is described 
in the early artificial intelligence classic by Feigenbaum 
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and Feldman (1963). Psychosocial models of individual and 
group behavior for performance analysis and prediction are 
described in Siegel and Wolf (1969). These citations are 
but a meager sample. 


5.2. Simulation As a Scholarly Activity 


Until the late-1960s, in the perceptions of RGS and 
REN, fundamental developments in simulation were readily 
accepted in the scientific literature. About this time an atti- 
tude of scholarly disrespect seemed to emerge. Many pro- 
fessionals in management science and operations research 
cast simulation as the “method of last resort” and expressed 
the view that “anyone could do it.” Unfortunately, the belief 
that simulation was simply a programming exercise led 
to that conviction becoming widespread among those who 
understood neither simulation nor computer programming. 
Did this pejorative view arise from the misuse of the 
term “simulation,” the far-reaching applications in so many 
diverse fields, or the preoccupation of OR and MS with 
mathematical sophistication? Or did it stem from a com- 
bination of these factors? The answer is not obvious, but 
the emergence of simulation departments (or areas) in the 
archival journals in the late 1970s gave evidence of a 
reputation regained. An increasing number of simulation 
researchers were finding outlets for quality publications. 


5.3. The Future of Simulation 


Computer graphics, virtual reality, and virtual environments 
are defining new vistas for simulation, but at the same time 
creating threats to overwhelm it. Entertainment uses and 
extensions of the technique offer financial inducements that 
are mind-boggling. Real-time and web-based models can 
expand and extend the impact far beyond its current level. 
At the same time, simulation-based acquisition and med- 
ical training applications impose requirements that appear 
daunting. Perhaps we are on the verge of achieving that 
which J. C. R. Licklider (1967) predicted some 35 years 
ago: 


In their dynamic form, however, computer-program 
models appeal to the recipient’s understanding directly 
through his perception of dynamic behavior. That mode 
of appeal is beyond the reach of ordinary documents. 
When we have learned how to take good advantage of it, 
it may—indeed, I believe it will—be the greatest boon 
to scientific and technical communication, and to the 
teaching and learning of science and technology, since 
the invention of writing on a flat surface. 
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